Skip to content

[2.7][Filesystem] Changed dumpFile to allow dumping to streams... #14580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
Closed
5 changes: 5 additions & 0 deletions src/Symfony/Component/Filesystem/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
CHANGELOG
=========

2.7.0
-----

* added tempNam() a stream aware version of tempnam()

2.6.0
-----

Expand Down
89 changes: 88 additions & 1 deletion src/Symfony/Component/Filesystem/Filesystem.php
Original file line number Diff line number Diff line change
Expand Up @@ -452,6 +452,63 @@ public function isAbsolutePath($file)
return false;
}

/**
* Creates a temporary file with support for custom stream wrappers.
*
* @param string $dir The directory where the temporary filename will be created.
* @param string $prefix The prefix of the generated temporary filename.
* Note: Windows uses only the first three characters of prefix.
*
* @return string The new temporary filename (with path), or false on failure.
*/
public function tempNam($dir, $prefix)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new feature, it should be documented as such.
But do we really want/require it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to add to symfony-docs if this feature is wanted.

Without a stream aware version of tempnam it is very difficult to write to a stream atomically. The path of least resistance becomes writing to the stream non-atomically using file_put_contents, circumventing the benefit of dumpFile. See #10018.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pleas also add a line in the CHANGELOG of the component.
I'm not convinced by this implementation. What about:

do {
    $tmpnam = $dir.'/'.$prefix.uniqid(mt_rand(), true);
} while (file_exists($tmpnam));

return $tmpnam;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicolas-grekas. Thanks for the feedback.

Change log: Will do.

Optimisation, agreed, speed more important than using OS's method of name generation. Will update.

Ta

{
$limit = 10;
$schemeSeparator = '://';
$scheme = $this->getScheme($dir);
$hierarchy = $this->getHierarchy($dir);

// If no scheme or scheme is "file" create temp file in local filesystem
if (false === $scheme || 'file' === $scheme) {
$tmpFile = tempnam($hierarchy, $prefix);
}

// If no scheme we're done, just return the file generated by tempnam
if (false === $scheme) {
return $tmpFile;
}

// If scheme is "file" add this back onto whatever tempnam created
if ('file' === $scheme) {
// Return the version with scheme
return $scheme.$schemeSeparator.$tmpFile;
}

// Loop until we create a valid temp file or have reached $limit attempts
for ($i = 0; $i < $limit; $i++) {

// Create a unique filename
$tmpFile = $dir.'/'.$prefix.uniqid(mt_rand(), true);

// If the file already exists restart the loop
// Use fopen instead of file_exists as some streams don't support stat
if (false !== @fopen($tmpFile, 'r')) {
continue;
}

// Create the file, if unsuccessful then stream is not writable so set to false
// Use file_put_contents instead of touch as it support stream wrappers
if (false === @file_put_contents($tmpFile, '')) {
return false;
}

return $tmpFile;

}

return false;
}

/**
* Atomically dumps content into a file.
*
Expand All @@ -472,7 +529,7 @@ public function dumpFile($filename, $content, $mode = 0666)
throw new IOException(sprintf('Unable to write to the "%s" directory.', $dir), 0, null, $dir);
}

$tmpFile = tempnam($dir, basename($filename));
$tmpFile = $this->tempNam($dir, basename($filename));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of creating a custom tempNam implementation, wouldn't it be better to bypass the temporary file creation and directly write to $filename, WHEN a stream wrapper is used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous comment regarding atomicity. If dumpFile was implemented in this way, I'm not sure there would be any benefit of using it over file_put_contents?


if (false === @file_put_contents($tmpFile, $content)) {
throw new IOException(sprintf('Failed to write file "%s".', $filename), 0, null, $filename);
Expand Down Expand Up @@ -501,4 +558,34 @@ private function toIterator($files)

return $files;
}

/**
* Gets the scheme part of a filename if present or false otherwise (e.g. file:///tmp -> file).
*
* @param string $filename The filename to be parsed.
*
* @return string|bool The filename scheme if present or false.
*/
private function getScheme($filename)
{
$schemeSeparator = '://';
$components = explode($schemeSeparator, $filename, 2);

return count($components) >= 2 ? $components[0] : false;
}

/**
* Gets the hierarchy part of a filename (e.g. file:///tmp -> /tmp).
*
* @param string $filename The filename to be parsed.
*
* @return string The filename hierarchy.
*/
private function getHierarchy($filename)
{
$schemeSeparator = '://';
$components = explode($schemeSeparator, $filename, 2);

return count($components) >= 2 ? $components[1] : $components[0];
}
}
4 changes: 4 additions & 0 deletions src/Symfony/Component/Filesystem/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ $filesystem->makePathRelative($endPath, $startPath);
$filesystem->mirror($originDir, $targetDir, \Traversable $iterator = null, $options = array());

$filesystem->isAbsolutePath($file);

$filesystem->tempNam($dir, $prefix);

$filesystem->dumpFile($file, $content);
```

Resources
Expand Down
83 changes: 83 additions & 0 deletions src/Symfony/Component/Filesystem/Tests/FilesystemTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -955,6 +955,66 @@ public function providePathsForIsAbsolutePath()
);
}

public function testTempNam()
{
$dirname = $this->workspace;

$filename = $this->filesystem->tempNam($dirname, 'foo');

$this->assertNotFalse($filename);
$this->assertFileExists($filename);
}

public function testTempNamWithFileScheme()
{
$scheme = 'file://';
$dirname = $scheme.$this->workspace;

$filename = $this->filesystem->tempNam($dirname, 'foo');

$this->assertNotFalse($filename);
$this->assertStringStartsWith($scheme, $filename);
$this->assertFileExists($filename);
}

public function testTempNamWithZlibScheme()
{
$scheme = 'compress.zlib://';
$dirname = $scheme.$this->workspace;

$filename = $this->filesystem->tempNam($dirname, 'bar');

$this->assertNotFalse($filename);
$this->assertStringStartsWith($scheme, $filename);
// Zlib stat uses file:// wrapper so remove scheme
$this->assertFileExists(str_replace($scheme, '', $filename));
}

public function testTempNamWithHTTPSchemeFails()
{
$scheme = 'http://';
$dirname = $scheme.$this->workspace;

$filename = $this->filesystem->tempNam($dirname, 'bar');

$this->assertFalse($filename);
}

public function testTempNamOnUnwritableFallsBackToSysTmp()
{
$scheme = 'file://';
$dirname = $scheme.$this->workspace.DIRECTORY_SEPARATOR.'does_not_exist';

$filename = $this->filesystem->tempNam($dirname, 'bar');

$this->assertNotFalse($filename);
$this->assertStringStartsWith(rtrim($scheme.sys_get_temp_dir(), DIRECTORY_SEPARATOR), $filename);
$this->assertFileExists($filename);

// Tear down
unlink($filename);
}

public function testDumpFile()
{
$filename = $this->workspace.DIRECTORY_SEPARATOR.'foo'.DIRECTORY_SEPARATOR.'baz.txt';
Expand Down Expand Up @@ -1009,6 +1069,29 @@ public function testDumpFileOverwritesAnExistingFile()
$this->assertSame('bar', file_get_contents($filename));
}

public function testDumpFileWithFileScheme()
{
$scheme = 'file://';
$filename = $scheme.$this->workspace.DIRECTORY_SEPARATOR.'foo'.DIRECTORY_SEPARATOR.'baz.txt';

$this->filesystem->dumpFile($filename, 'bar', null);

$this->assertFileExists($filename);
$this->assertSame('bar', file_get_contents($filename));
}

public function testDumpFileWithZlibScheme()
{
$scheme = 'compress.zlib://';
$filename = $this->workspace.DIRECTORY_SEPARATOR.'foo'.DIRECTORY_SEPARATOR.'baz.txt';

$this->filesystem->dumpFile($filename, 'bar', null);

// Zlib stat uses file:// wrapper so remove scheme
$this->assertFileExists(str_replace($scheme, '', $filename));
$this->assertSame('bar', file_get_contents($filename));
}

public function testCopyShouldKeepExecutionPermission()
{
$sourceFilePath = $this->workspace.DIRECTORY_SEPARATOR.'copy_source_file';
Expand Down